Papers with Large Multimodal Model
WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing web agents only handle one input modality and are evaluated only in simplified web simulators or static web snapshots, greatly limiting their applicability in real-world scenarios. |
| Approach: | They propose a large multimodal model-powered web agent that can complete user instructions end-to-end by interacting with real-world websites. |
| Outcome: | The proposed agent achieves 59.1% task success rate, surpassing both GPT-4 and WebVoyager setups. |
BiMediX2 : Bio-Medical EXpert LMM for Diverse Medical Modalities (2025.findings-emnlp)
Copied to clipboard
Sahal Shaji Mullappilly, Mohammed Irfan Kurpath, Sara Pieri, Saeed Yahya Alseiari, Shanavas Cholakkal, Khaled M Aldahmani, Fahad Shahbaz Khan, Rao Muhammad Anwer, Salman Khan, Timothy Baldwin, Hisham Cholakkal
| Challenge: | BiMediX2 is a bilingual (Arabic-English) large multimodal model that supports text-based and image-based medical interactions. |
| Approach: | They introduce BiMediX2, a bilingual (Arabic-English) Bio-Medical EXpert Large Multimodal Model that supports text-based and image-based medical interactions. |
| Outcome: | The model outperforms existing models by over 9% in English and more than 20% in Arabic evaluations. |